Crucial Factors in Cooperative Multirobot Learning

Cooperative decentralized multirobot learning refers to the use of multiple learning entities to learn optimal solutions for an overall multirobot system. We demonstrate that traditional single-robot learning theory can be successfully used with multirobot systems, but only under certain conditions. The success and the effectiveness of single-robot learning algorithms in multirobot systems are potentially affected by various factors that we classify into two groups: the nature of the robots and the nature of the learning. Incorrect set-up of these factors may lead to undesirable results. In this paper, we systematically test the effect of varying five common factors (model of the value function, reward scope, delay of global information, diversity of robots’ capabilities, and number of robots) in decentralized multirobot learning experiments, first in simulation and then on real robots. The results show that three of these factors (model of the value function, reward scope, and delay of global information), if set up incorrectly, can prevent robots from learning optimal, cooperative olutions. s K eywords: multirobot learning, distributed reinforcement learning, cooperative robots